Vehicles often encounter obstacles while driving. Obstacles include potholes, debris in the road, missing lane markers, or any other thing that could cause a vehicle to alter its present operating conditions. Some vehicles may operate autonomously. Autonomous mode can include a system that fully controls all aspects of the operation of the vehicle. Autonomous mode can also include systems that assist a driver, for example adaptive cruise control, parking assist, lane departure warning systems, blind spot monitors, or any other advanced driver assistance system (ADAS).